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A METHOD FOR TRACKING PARTICLES AND LIFE FORMS IN THREE 

DIMENSIONS AND IN TIME 

Cross-Reference to Related Application 

This application claims priority from U.S. Provisional Patent Application Serial No. 
5 60/429,355, filed November 27, 2002, the entirety of which is incorporated herein by 
reference. 

Field of Invention 

The present invention relates to the field of holographic microscopy. In particular, to a 
method for tracking the trajectories of particles in three dimensions and in time using 
1 0 digital in-line holography. 

Background 

The efficient recording of the trajectories of micron size particles that are moving 
throughout a three dimensional (3-D) space has been an important problem in several 
branches of science such as colloidal suspensions, the motion of algae or larvae in water, 
1 5 the motion of bacteria around and in cells, and the characterization of marine particulates. 

One approach to tracking such particles has been the use of compound light microscopy. 
Conventional compound light microscopy can give high-resolution information about an 
object but only in a single focal plane. Therefore, compound light microscopy is 
effectively limited to two dimensional tracking of particles. 

20 Digital in-line holography (DIH) offers a rapid and efficient approach to construct high- 
contrast 3-D images of a sample volume from a single hologram. An exemplary 
approach to DIH is described in U.S. Patent No. 6,411,406 issued June 25 th , 2002 to H. 
Juergen Kreuzer which is incorporated in its entirety herein by reference. In digital in- 
line holography a spherical wave, emanating from a "point" source of linear dimensions 

25 of the order of the wavelength, illuminates an object, typically at a distance of a few 
thousand wavelengths from the source, and forms a highly magnified diffraction pattern 
on a screen much further away. Details of DIH and a thorough discussion of its history 
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and potential have been presented, in a number of publications together with earlier 
results in such diverse areas as cell biology, micro-particle imaging and tracking, and 
polymer crystallization. 

Holography is a two-step process: first, a hologram must be recorded, and second, it must 
5 be reconstructed to yield an "image" of the object. In DIH the hologram is recorded by a 
detector array, such as a charge-coupled device (CCD) camera for detecting photons, and 
the frame (i.e. the detector recorded hologram) is then captured by a computer in which 
the reconstruction is done using numerical means. The role of reconstruction is to obtain 
the three-dimensional structure of the object from the two-dimensional hologram on the 
10 screen (i.e. the detector array), or, in physical terms, to reconstruct the wave front at the 
object. This can be achieved via a Kirchhoff-Helmholtz transform such as that 
represented in Equation 1 . 

S 

In the Kirchhoff-Helmholtz transform represented in Equation 1, the integration extends 
15 over the two dimensional surface of the screen (assumed to be perpendicular to the 
optical axis) with coordinates £=(X,Y,L), where L is the distance from the source 
(pinhole) to the center of the screen (detector array) and I(£ ) is the contrast image 
(hologram) on the screen, obtained by subtracting the images with and without the object 
present. The function K(r) is significantly structured and differs from zero only in the 
20 space region occupied by the object. By reconstructing the wave front K(r) on a number 
of planes at various distances from the source in the vicinity of the object, a three- 
dimensional image can be built up from a single two-dimensional hologram. K(r) is a 
complex function and one usually plots its magnitude to represent the object, although 
phase images are also available. For the numerical implementation of the transform a 
25 fast algorithm that evaluates K(r) without any approximations can be used. The algorithm 
employs a coordinate transformation that transforms the integral into a convolution that is 
solved by three consecutive Fast Fourier Transforms. 
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In holography, the term 'reconstruction' is used to describe obtaining the function K(r) 
from the hologram. The plot of \K(r)\ on a two-dimensional plane perpendicular to the 
optical axis, is called a two dimensional (2-D) holographic reconstruction, is equivalent 
to a single in-focus image taken in a conventional compound microscope. In DIH a stack 
5 of 2-D holographic reconstructions can be generated from a single hologram. Combining 
such a stack results in a three-dimensional image of the object; this latter step is usually 
referred to as 3-D reconstruction. 

The input to the 3-D reconstruction as represented, for example, by equation (1) is the 
contrast image for a perfectly spherical incoming wave. Perfecting this image is the 

10 hardest part in the practical implementation of DIH. The normal procedure for 
generating the contrast image is as follows: (i) Record digitally the hologram of the 
object, i.e. the intensity matrix, I„ m , recorded on the detector array, where n and m 
enumerate pixels in the x- and y-axis. (ii) Remove the object and record digitally the 
intensity matrix of the illuminating laser, (iii) Subtract the results of (ii) from the results 

1 5 of (i) to numerically construct the contrast image, corrected for the intensity variations in 
the primary laser beam. Using this procedure almost all imperfections in the laser source 
are eliminated. Indeed, this procedure minimizes the quality requirements on the laser 
itself, as long as the laser is sufficiently stable to identically capture both images. 

In many situations it may not be possible or convenient to remove the object from the 
20 optical path in order to construct the contrast image. This is clearly the case when the 
object is in motion and it is desired to record the time evolution of the object's trajectory. 

What is needed is an effective approach for tracking the trajectories of particles and life 
forms that are moving throughout a three dimensional space. 

Summary of Invention 

25 The present invention is directed to a method for tracking particles and life forms in three 
dimensions and in time. The present invention applies a numerical reconstruction 
approach to digital in-line holographic microscopy images in order to generate a time 
sequence hologram representing the trajectory of objects such as particles and life forms. 
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By subtracting consecutive (in time) holograms of a particle suspension and then adding 
the resulting differences, a final data set (hologram) is constructed that contains the time 
evolution of the object trajectory free from spurious background interference effects. 

A method according to the present invention provides for the removal of a) distortions or 
5 deviations from a theoretically perfect spherical wave front from an illuminating laser, b) 
distortions due to an imperfect pinhole, and c) other optical/image interference caused by 
dust, background, noise etc. This requires the generation of two holographic images, one 
with a subject present, the second with the subject either in a new position or absent. The 
first image is then subtracted from the second such as to generate a 'background free' 
10 image containing only (i.e. isolating) the subject, which can then be reconstructed via 
mathematical formulae such as, for example, those described in U.S. Patent No. 
6,41 1,406 issued June 25 th , 2002 to H. Juergen Kreuzer. 

A method according to the present invention further provides for tracking of the subject 
via the repeated subtraction of every second image from a previous image in a series of N 
15 time-spaced images, such as to generate a series of N/2 time-spaced holographic images 
(e.g. (h, - h 2 ), (h 3 - 114),..., (h n -i - h n )) that contain only the subject. 

A method according to the present invention further provides for adding (combining) of 
the series of N/2 time-spaced images together to generate a single holographic image that 
contains the images of the subjects at all positions in time represented in the full series of 

20 N time-spaced images. Numerical reconstruction is then performed on the single 
combined holographic image at each of a plurality of reconstruction planes. The 
reconstruction for a given reconstruction plane results in a view of the positions of all 
subjects, in all time-spaced images, that were ever present in that plane. For a subject 
that, for example, moves only horizontally within the reconstruction plane in the time- 

25 spaced images, the reconstructed image will effectively display its trajectory (i.e. its 
successive locations over the full time-period represented by the N time-spaced images). 

Alternatively, a method according to the present invention can use algorithms, for 
generating the series of time spaced holograms, other than the subtraction of every 
second image (i.e. (h, - h 2 ), (h 3 - to),..., (hn-i - h„)) algorithm. For example, a single 
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image can be subtracted from a succession of time spaced images (e.g. (h 2 - hi), (h 3 - hi), 
(h4 - hi)... (h n - hi)) to derive the series of time spaced images. Such an embodiment is 
particularly useful when the subject is moving slowly in which case the embodiment in 
which every second image is subtracted from the preceding image could cause the 
5 elimination of a part or all of the subject in a resulting difference image. 

In accordance with one aspect of the present invention, method for tracking the trajectory 
in three-dimensions and in time of an object in a sample volume comprising the steps of: 
a) recording a time-spaced sequence of digital in-line holograms, generating a sequence 
of N holograms; b) subtracting from a first hologram a second hologram in each 
1 0 successive pair of the sequence of N holograms to generate N/2 difference holograms; c) 
summing the N/2 difference holograms to generate a summed hologram; and d) 
reconstructing images of the object at a plurality of depths into the sample volume 
representing the trajectory of the object. 

In accordance with another aspect of the present invention, a method for tracking the 
1 5 trajectory in three-dimensions and in time of an object in a sample volume comprising the 
steps of: a) recording a time-spaced sequence of digital in-line holograms, generating a 
sequence of N holograms; b) subtracting a first hologram from each of the remaining 
holograms of the sequence of N holograms to generate N-l difference holograms; c) 
summing the N-l difference holograms to generate a summed hologram; and d) 
20 reconstructing images of the object at a plurality of depths into the sample volume 
representing the trajectory of the object. 

In accordance with still another aspect of the present invention, a method for tracking the 
trajectory in three-dimensions and in time of an object in a sample volume comprising the 
steps of: a) recording a time-spaced sequence of digital in-line holograms, generating a 
25 sequence of N holograms; b) subtracting a first hologram from each of the remaining 
holograms of the sequence of N holograms to generate N-l difference holograms; c) 
reconstructing an image of the object at a depth into the sample volume for each of the N- 
1 difference holograms, generating N-l subject images; and d) combining the N-l subject 
images to generate a 3-D image representing the trajectory of the object. 
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In accordance with yet another aspect of the present invention, a method for tracking the 
trajectory in three-dimensions and in time of a plurality of objects in a sample volume 
comprising the steps of: a) recording a digital in-line hologram as a first series of pixels 
in a buffer; b) for a subsequent time-spaced sequence of digital in-line holograms, 

5 recording each hologram represented as a second series of pixels by: i) subtracting each 
pixel in the second series of pixels from a corresponding pixel in the buffer, for a 
hologram having an even ordinal number in the sequence of digital in-line holograms and 
ii) adding each pixel in the second series of pixels to a corresponding pixel in the buffer, 
for a hologram having an odd ordinal number in the sequence of digital in-line 

10 holograms; and c) reconstructing images of the object at a plurality of depths into the 
sample volume representing the trajectory of the object. 

Other aspects and features of the present invention will become apparent to those 
ordinarily skilled in the art to which it pertains upon review of the following description 
of specific embodiments of the invention in conjunction with the accompanying figures. 

15 Brief Description of Drawings 

The present invention will be described in conjunction with the drawings in which: 

Figures 1 A-D illustrate holograms and reconstructed trajectories, according to the present 
invention, of the alga Tetraselmis in sea-water using a DIH approach. 

Figure 2A-B illustrate reconstructed trajectories, according to the present invention, of 
20 the alga Tetraselmis in seawater from four individual holograms using a DIH approach. 

Figure 3 is a flow chart of the steps in an exemplary embodiment of a method according 
to the present invention. 

Detailed Description 

Applying a digital in-line holography approach using a detector array having a fast 
25 capture rate, it is possible to record digitally the motion of particles in a sample volume 
from their successive positions in time. Subsequently images at selected depths can be 
reconstructed so that the trajectory and speed of a collection of particles can be captured 
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as 3-D data sets. To illustrate a method according to the present invention, holographic 
images were recorded (see Fig. 1A described below) of a live culture of the single-celled 
marine flagellate Tetraselmis, contained in a 1 mm thick layer of salt water between a 
microscope slide and a cover slip. These algae are flattened and propel themselves by 
5 four anterior flagella. 

Figures 1 A-D illustrate holograms and reconstructed trajectories, according to the present 
invention, of the alga Tetraselmis in sea-water using a DIH approach (Blue laser, 1 urn 
pinhole, source to sample distance 2.5 mm, source to CCD camera distance 20 mm.). 
Figure 1A illustrates a single hologram of three algae (T1-T3) with an inset bright-field 

10 image of single alga, showing flagella (arrowhead); (Zeiss, Plan-Neofluar 100 X /1.30 
oil). Figure IB illustrates a sum of 10 difference holograms (from a total of 20) of the 
trajectories of the algae in Figure 1A. Figure 1C illustrates a reconstruction of the 
summed hologram of Figure IB in one plane with the trajectory of alga Tl in focus. 
Figure ID illustrates a reconstruction from the hologram of Figure IB in a plane 500 urn 

15 above that in Figure 1C. Arrows indicate the directions of algal motion. 

Figure 2A-B illustrate reconstructed trajectories, according to the present invention, of 
the alga Tetraselmis in seawater from four individual holograms using a DIH approach 
(Blue laser, 1 urn pinhole, source to CCD camera distance 3.05 cm, reconstruction area 1 
x 1 mm 2 ). In Figure 2A arrows mark three algae that are in sharp focus in a plane 2.4 mm 

20 from the pin-hole. In Figure 2B arrows mark four algae in sharp focus in a plane 6.7 mm 
from the pin-hole. In both Figures 2A and 2B several less sharply reconstructed algae 
trajectories are also visible. Sets of closely spaced reconstruction planes thus allow the 
determination of algae density throughout volumes several cm 3 in size. Analysis of the 
sample illustrated in Figures 2A-B through multiple reconstructions has shown that this 

25 sample contained 96 algae in a volume of about 1 mm 3 of sea-water. 

Figure 3 is a flow chart of the steps 100 in an exemplary embodiment of a method 
according to the present invention. To obtain high-resolution DIH reconstruction images 
of the movement trajectories of an object (e.g. a particle or a life form), an exemplary 
embodiment of a method according to the present invention begins with a step 110 of 
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recording a sequence N of time-spaced holograms (h,,... ,h N ). This is followed by a step 
120 of removing undesired background effects (such as the set of large concentric rings 
in Fig. 1A) thereby isolating the object by, for example, subtracting consecutive 
hologram pairs, pixel by pixel, to generate difference holograms, i.e. (hi-h 2 ), (h 3 -ti4), etc. 

5 Next in a step 130 the resultant difference holograms (i.e. (hi,-h 2 ), (h 3 -li4), etc.) are 
summed into a single summed hologram. The summed hologram contains all holograms 
in the sequence with the sign alternated for each successive (alternate) hologram. 
Preferably the summed hologram has the same size as any single original hologram. 
Subtracting alternate holograms preferably ensures that the dynamic range of the 

10 numerical processing means is not exceeded and that only the object-related-information 
is retained. In a step 140 the summed hologram is reconstructed using, for example, a 
Kirchhoff-Helmholtz transform (see Equation 1) to obtain images at reconstruction 
planes at different depths in the sample volume. Such images show the sequential 
positions at successive recording times of the object contained in the sample volume. 

15 Figs. 1C and ID show the trajectories of three algae in two such reconstructions in two 
planes separated by 500 um. Twenty sequential positions clearly define the trajectory of 
alga Tl (Fig. 1A), which swims in a zigzag fashion more or less confined to a plane 
parallel to the glass slide. It also rotates about the long axis, the flattened shape 
alternately imaged edge on (triangle: Fig. 1C) or en face (the two preceding images). 

20 Access to 3-D information in holographic imaging, according to the present invention, is 
particularly well illustrated by alga T3 (Fig.lC). This alga comes to lie in the 
reconstruction plane but earlier images are progressively out of focus, indicating that the 
direction of motion had a large component perpendicular to the reconstruction plane. To 
obtain the complete trajectory, further reconstructions from the same hologram file can 

25 be made in as many planes as necessary without the need for recording further 
holograms. Reconstructed in one plane (Fig. 1C), the trajectory of alga T2 is visible, 
only severely out of focus, but becomes clearly visible when the hologram in Fig. IB is 
reconstructed in a plane that lies 500 um higher, Fig. ID. From the reconstructions an 
average algal swimming speed of approximately 150 um/s was obtained. 
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A method according to the present invention is not limited to the sampling of small 
volumes. Figures 2A-B show an example where algae were imaged in several cm of 
water contained in a rectangular spectrometer optical cell. Figures 2A-B illustrate how a 
method according to the present invention can be used to determine the distribution of 

5 moving particles in a suspension. For the images in Figures 2A-B only four separate 
holograms were recorded. Figure 2A shows the short trajectories of several algae 
(marked by arrows) that lie in the reconstruction plane. Other, out of focus, trajectories 
are also visible in the image. Representation of a particle by a short trajectory instead of 
a single spot made it easier to identify the algae. In Figure 2B the same data set shows 

1 0 algae in a plane 4.3 mm away from the plane of Figure 2 A. Volumes of several cm 3 can 
be examined easily in this manner. 

In an exemplary embodiment of a method according to the present invention removal of 
background effects and reconstruction of a summed hologram are easily accomplished so 
that high-resolution tracking of many particles in 4-D (3-D and time) from just a single 
1 5 hologram data set can be performed. Since resolutions on the order of the wavelength of 
light have been achieved with DIH, tracking of organisms as small as bacteria, the motion 
of plankton in water and micro-particles in fluid flow have also been performed. At 
lower resolution, the aerial trajectories of flying insects should be possible. 

When a holographic image is subtracted from the previous image in the time-spaced 
20 series e.g. (h 3 - lu), the resultant single image effectively contains two images of each 
single subject, one positive and one negative, with any background or image data 
common to both images removed. The resulting N/2 images have N subject positions 
recorded in the N/2 images. 

In the above described algorithm (i.e. subtraction of every other image), if a subject does 
25 not move or change between the two images (i.e. is identical in both images) it will be 
removed. That is, there will be no image of the subject shown at that position in the 
resultant reconstructed track image. In an alternative embodiment of a method according 
to the present invention, this limitation can be overcome by subtracting a later image in 
the sequence instead of the immediately next image, for example (hi-lu), (h 2 -h 5 ), 
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(h 3 - h6>, (h 7 - h 10 )> (h 8 - hi j), (h 9 - h !2 ). The increased time spacing between the subtracted 
images increases the possibility of movement between the subtracted images. The N/2 
images are added together to derive the single holographic image. Reconstruction can be 
performed at each depth to derive the track images at each depth. In practice, the first 
5 position of a subject is always seen and known, and positions where a subject is 
stationary can be inferred from the multiple positions shown for other subjects. 

In an alternative embodiment of a method according to the present invention, a single 
image can be subtracted from a succession of time spaced images (e.g. (h 2 - h|), (h 3 - hi), 
(h4 - hi)... (h n - hi)) to derive the series of time spaced images. 

10 In another alternative embodiment of a method according to the present invention, 
reconstruction can be preformed, for a given depth, on each of the plurality of difference 
holograms to generate subject images. The subject images can then be combined 
together to generate a 3-D image representing the sequential positions at successive 
recording times of the object contained in the sample volume. 

15 The summing of the successive subtracted pairs causes the individual pixel values in the 
final single summed hologram to remain 'within bounds' (i.e. prevents data overflow e.g. 
(h, - h 2 ) + (h 3 - h4) + (h 5 - h 6 ) + (h 7 - h 8 ) = h, - h 2 + h 3 - \u + h 5 - h 6 + h 7 - h 8 ). In an 
alternative embodiment of a method according to the present invention, each successive 
image can be 'added' to a summed image with a sign reversal for every other 'addition'. 

20 It is therefore not necessary to retain the previous image for summing purposes - the 
pixels of the next image in the series are merely added to or subtracted from the pixels of 
the summed image. 

A method, according to the present invention, for tracking the three-dimensional 
movement of particles and minute life forms in time provides a single summed 
25 holographic image, containing the full 3-D movement of many objects in a sample 
volume over the same time period. This facilitates the study of how interactions and 
relationships between particles affect the trajectories and movements. It is also extremely 
useful for the study of such subjects as bacteria attacking cells as the full movement of 



10 



48734 



the bacteria in three-dimensions can be studied in the one sequence of time-spaced 
images all summed to create one holographic image. 

It will be apparent to one skilled in the art that numerous modifications and departures 
from the specific embodiments described herein may be made without departing from the 
5 spirit and scope of the present invention. 
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